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DETAILED ACTION 
Response to Amendment 

1 . Receipt of Applicant's Amendment, filed 1 2/1 9/2007 is acknowledged. 
Claims 1-14 and 16-17 have been amended. 



Double Patenting 

2. The provisional non-statutory double patenting rejection will be held in abeyance 
until such time as a patent issues based upon one of these applications, at which time a 
substantive response will be provided if the rejection under the judicially created 
doctrine of obviousness-type double patenting is maintained. 



Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
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consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 



Claims 1-11 and 13-18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Rothmuller et al. (Rothmuller hereinafter) (International Pub No. 
WO 02/057959 A2) in view of Parker et al. (Parker hereinafter) (U.S. Patent No. 
2003/0009493). 

With respect to claim 1 , Rothmuller teaches a computer program product 
comprising a computer readable storage medium having computer-readable 
program Instructions embodied in the medium, the computer-readable program 
instructions comprising: 

"first instructions configured, when executed, to generate a media view" as 
figure 1, Image area 100 provides a media view (Rothmuller Figure 1) "that provides 
access to digital media files and associates digital media files with a predefined 

time" as when the objects to be search for are photos, these search criteria can 
include, but are not limited to, the date and time the photos were taken, textual 
information that is associated with the photos such as the names of the people who are 
in the photos or the places or events where the photos were taken, designations of the 
photos as favorite photos, and designation of the photos as photos that have been 
printed, shared with others, or archived on a certain date (Rothmuller Page 2, Lines 
13-19). These media files are being accessed based on the association of these media 
files with respect to their data and time. 
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"second instructions configured, when executed, to generate a time bar 
that divides time into segments, each segment of time depends upon the amount 
of media files associated with the respective segment of time" as the temporal 
metadata associated with the photos can be used to present a histogram of photos in 

the form of a timeline 250 shown In fig 1 . The timeline 250 can show the number of 
photos take as function of time over some period of time that can rage from the time the 
first photo in the database was taken to the present. The timeline 250 can be used by 
itself, or with other tags 350 to specify the criteria used to search for matching photos. 
The timeline Includes adjustable time bands 251 that can be moved to allow timeline 
250 to specify the time period that is used to find matching photos (Rothmuller Page 7, 
Lines 27-31 & Page 8, Lines 1-3). Examiner interprets the segments as time bands 
having a size/time period and media files associated with that size. 

The timeline 250 can be used by itself to find all photos taken between Jan. 1 , 
2000 and Feb. 28, 2000 by moving the adjustable time bands 251 to these two 
respective dates (Rothmuller Page 8, Lines 8-10 & figure 1 and 3). In figure 3 there 
are segments of time and their size depends upon the amount of files for that time 
segment. 

Rothmuller teaches the elements of claim 1 as noted above but does not 
explicitly teaches "bar divided into segments of unit time and having length along 
the time bar." 

However, Parker teaches "bar divided into segments of unit time and having 
length along the time bar" as he number of histogram bins that can be clearly 
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displayed and the desired start and end date of the histogram timeline, new date bin 
sizes are calculated. In step 238, a new histogram timeline is generated with the 
modified bin sizes. Finally, in step 240, the modified histogram timeline with more 
visible bin size is displayed (Parker Paragraph 0048). Parker's timeline is being divided 
into units of time as being days, months, or years. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teaching of the cited references because Parker's 
teaching would have allowed Rothmuller to provide an effective method for organizing 
representations of digital multimedia objects to facilitate the selection of desired digital 
multimedia objects by selecting a portion of the histogram timeline for viewing such 
thumbnail representations of visual digital objects corresponding to such selected 
portion. 

These references disclose the claimed invention except for segments of time 
having length along time bar (horizontally) which depends on the amount of files 
associated with that length. Prior art teaches vertical bars, where bars represent the 
amount of media files associated with that time unit. It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify the vertical 
graphs for a specific time segment to horizontal graph along the time bar, since it has 
been held that the provision of adjustability, where needed, involves only routine skill in 
the art. In re Stevens. 101 USPQ 284 (CCPA 1954). 

Further, It would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify the vertical graphs for a specific time segment to 
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horizontal graph along the time bar, since it has been held that a mere reversal of the 
essential working parts of a device involves only routine skill in the art. In re Einstein, 8 
USPQ 167. 

With respect to claim 2, Rothmuller teaches "the computer program product 
of claim 1, wherein the second instructions configured to generate a time bar 
include instructions configured to generates selectable segments of unit time" as 
the timeline includes adjustable time bands 251 that can be moved to allow timeline 250 
to specify the time period that is used to find matching photos (Rothmuller Page 8, 
Lines 2-3). Examiner interprets selectable as adjustable. 

With respect to claim 3, Rothmuller teaches "the computer program product 
of claim 1, wherein the second instructions configured to generate a time bar 
include instructions configured to generate segments of unit time periods chosen 
from the group consisting of a year, a month, a week and a day" as the timeline 
250 can be used by itself to find all photos taken between Jan 1 , 2000 and Feb 28, 
2000 by moving the adjustable time bands 251 to these two respective dates 
(Rothmuller Page 8, Lines 8-10). 

Claim 14 is essentially the same as claim 3 except it recites the claimed invention 
as a method and is rejected for the same reasons as applied herein above. 
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With respect to claim 4 Rothmuller teaches "the computer program product 
of claim 1, wherein the second instructions configured to generate a time bar 
include instructions configured to generate a segment of unit time that indicates 
the amount of media files in the time segment" as the temporal metadata associated 
with the photos can be used to present a histogram of photos in the form of a timeline 
250 shown in fig 1 . The timeline 250 can show the number of photos take as function of 
time over some period of time that can rage from the time the first photo in the database 
was taken to the present. The timeline 250 can be used by itself, or with other tags 350 
to specify the criteria used to search for matching photos. The timeline includes 
adjustable time bands 251 that can be moved to allow timeline 250 to specify the time 
period that is used to find matching photos (Rothmuller Page 7, Lines 27-31 & Page 8, 
Lines 1-3). 

With respect to claim 5, Rothmuller teaches "the computer program product 
of claim 4, wherein the second instructions configured to generate a time bar 
includes instructions configured to generate a segment of time that indicates the 
amount of media items in the segment based on the size of the segment" as the 

temporal metadata associated with the photos can be used to present a histogram of 
photos in the form of a timeline 250 shown in fig 1 . The timeline 250 can show the 
number of photos take as function of time over some period of time that can rage from 
the time the first photo in the database was taken to the present. The timeline 250 can 
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be used by itself, or with other tags 350 to specify the criteria used to search for 
matching photos. The timeline includes adjustable time bands 251 that can be moved 
to allow timeline 250 to specify the time period that is used to find matching photos 
(Rothmuller Page 7, Lines 27-31 & Page 8, Lines 1-3). 

With respect to claim 6, Rothmuller teaches "the computer program product 
of claim 4, wherein the second instructions configured to generate a time bar 
includes instructions configured to generate a segment of time that indicates the 
amount of media items in the segment unit based on the color of the segment" as 

when tag searches are conducted in conjunction with timeline 250, the timeline displays 
the total number of photos in the database per unit time period in a first color which may 
be a solid color, and the total number of photos in the database that match the tagged 
search criteria as "best" or "close" matches in a second color which may be a hatched 
pattern or color (Rothmuller Page 8, Lines 21-26). 

With respect to claim 7, Rothmuller teaches "the computer program product 
of claim 1, wherein the second instructions configured to generate a time bar 
additionally includes instructions configured to generate a time handle that 
allows for periods of time to be scrolled" as figure 1 and figure 3, which shows the 
time handle/arrows to scroll the time periods (Rothmuller Figure 1 & 3). 
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With respect to claim 8, Rothmuller teaches "the computer program product 
of claim 1, wherein the first instructions further include instructions configured to 
generate digital media files with a predefined time based upon information 
associated with the digital media file" as when the objects to be search for are 

photos, these search criteria can include, but are not limited to, the date and time the 
photos were taken, textual information that is associated with the photos such as the 
names of the people who are in the photos or the places or events where the photos 
were taken, designations of the photos as favorite photos, and designation of the photos 
as photos that have been printed, shared with others, or archived on a certain date 
(Rothmuller Page 2, Lines 13-19). These media files are being accessed based on the 
association of these media files with respect to their data and time. 

Claim 15 is essentially the same as claim 15 except it recites the claimed 
invention as a method and is rejected for the same reasons as applied herein above. 

With respect to claim 9, Rothmuller teaches "the computer program product 
of claim 1, further including third instructions configured to generate a calendar 
view that represents time in calendar format and associates events with 
respective periods of time" as in addition to timelines, the temporal distribution of 

objects in the database can be represented in a calendar view such that the days of the 
calendar indicate the number of objects having metadata associated with a given day of 
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the week in a given week of the month (Rothmuller Page 3, Lines 10-16). In addition 
tags in the events category can include pre-defined calendar event such as New Years 
Eve, and customized calendar events such as birthdays and anniversaries (Rothmuller 
Page 5, Lines 21-23). 

With respect to claim 10, Rothmuller teaches "the computer program product 
of claim 9, wherein the first instructions configured to generate a media view that 
provides access to digital media files and associates digital media files with a 
predefined time, associates digital media files with a past predefined time and 
wherein the third instructions for generating a calendar view that represents time 
in calendar format and associates events with respective periods of time, 
associates events with respective future periods of time" as figure 1 , Image area 
100 provides a media view (Rothmuller Figure 1). When the objects to be search for 
are photos, these search criteria can include, but are not limited to, the date and time 
the photos were taken, textual information that is associated with the photos such as the 
names of the people who are in the photos or the places or events where the photos 
were taken, designations of the photos as favorite photos, and designation of the photos 
as photos that have been printed, shared with others, or archived on a certain date 
(Rothmuller Page 2, Lines 13-19). These media files are being accessed based on the 
association of these media files with respect to their data and time. 

In addition to timelines, the temporal distribution of objects in the database can 
be represented in a calendar view such that the days of the calendar indicate the 
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number of objects having metadata associated with a given day of the week in a given 
week of the month (Rothmuller Page 3, Lines 10-16). In addition tags in the events 
category can include pre-defined calendar event such as New Years Eve, and 
customized calendar events such as birthdays and anniversaries (Rothmuller Page 5, 

Lines 21-23). 

With respect to claim 1 1 , Rothmuller teaches an appratus comprising: 

"a processing unit that executes computer-readable program instructions 
embodied in a computer readable storage medium, the computer-readable 
program instructions comprising" as a programmable processor executing a 
program of instruction to perform functions of the invention by operating on input data 
and generating output (Rothmuller Page 12, Lines 19-21). 

"first instructions configured, when executed, to generate a media view" as 
figure 1, Image area 100 provides a media view (Rothmuller Figure 1) "that provides 
access to digital media files and associates digital media files with a predefined 

time" as when the objects to be search for are photos, these search criteria can 
include, but are not limited to, the date and time the photos were taken, textual 
information that is associated with the photos such as the names of the people who are 
in the photos or the places or events where the photos were taken, designations of the 
photos as favorite photos, and designation of the photos as photos that have been 
printed, shared with others, or archived on a certain date (Rothmuller Page 2, Lines 
13-19). These media files are being accessed based on the association of these media 
files with respect to their data and time. 
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"second instructions configured, when executed, to generate a time bar 
that divides time into segments, each segment of time depends upon the amount 
of media files associated with the respective segment of time" as the temporal 
metadata associated with the photos can be used to present a histogram of photos in 

the form of a timeline 250 shown in fig 1 . The timeline 250 can show the number of 
photos take as function of time over some period of time that can rage from the time the 
first photo in the database was taken to the present. The timeline 250 can be used by 
itself, or with other tags 350 to specify the criteria used to search for matching photos. 
The timeline includes adjustable time bands 251 that can be moved to allow timeline 
250 to specify the time period that is used to find matching photos (Rothmuller Page 7, 
Lines 27-31 & Page 8, Lines 1-3). Examiner interprets the segments as time bands 
having a size/time period and media files associated with that size. 

The timeline 250 can be used by itself to find all photos taken between Jan. 1 , 
2000 and Feb. 28, 2000 by moving the adjustable time bands 251 to these two 
respective dates (Rothmuller Page 8, Lines 8-10 & figure 1 and 3). In figure 3 there 
are segments of time and their size depends upon the amount of files for that time 
segment. 

Rothmuller teaches the elements of claim 1 1 as noted above but does not 
explicitly teaches "bar divided into segments of unit time and having length along 
the time bar." 

However, Parker teaches "bar divided into segments of unit time and having 
length along the time bar" as he number of histogram bins that can be clearly 
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displayed and the desired start and end date of the histogram timeline, new date bin 
sizes are calculated. In step 238, a new histogram timeline is generated with the 
modified bin sizes. Finally, in step 240, the modified histogram timeline with more 
visible bin size is displayed (Parker Paragraph 0048). Parker's timeline is being divided 
into units of time as being days, months, or years. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teaching of the cited references because Parker's 
teaching would have allowed Rothmuller to provide an effective method for organizing 
representations of digital multimedia objects to facilitate the selection of desired digital 
multimedia objects by selecting a portion of the histogram timeline for viewing such 
thumbnail representations of visual digital objects corresponding to such selected 
portion. 

These references disclose the claimed invention except for segments of time 
having length along time bar (horizontally) which depends on the amount of files 
associated with that length. Prior art teaches vertical bars, where bars represent the 
amount of media files associated with that time unit. It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify the vertical 
graphs for a specific time segment to horizontal graph along the time bar, since it has 
been held that the provision of adjustability, where needed, involves only routine skill in 
the art. In re Stevens. 101 USPQ 284 (CCPA 1954). 

Further, It would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify the vertical graphs for a specific time segment to 
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horizontal graph along the time bar, since it has been held that a mere reversal of the 
essential working parts of a device involves only routine skill in the art. In re Einstein, 8 
USPQ 167. 

Claim 13 is essentially the same as claim 1 1 except it recites the claimed 
invention as a method and is rejected for the same reasons as applied herein above. 

With respect to claim 16, Rothmuller teaches a method comprising: 

"providing the user of a digital device a display of a time bar and a media 
view that represents media files in association with a predefined time" as the 

temporal metadata associated with the photos can be used to present a histogram of 
photos in the form of a timeline 250 shown in fig 1 . The timeline 250 can show the 
number of photos take as function of time over some period of time that can rage from 
the time the first photo in the database was taken to the present. The timeline 250 can 
be used by itself, or with other tags 350 to specify the criteria used to search for 
matching photos. The timeline includes adjustable time bands 251 that can be moved 
to allow timeline 250 to specify the time period that is used to find matching photos 
(Rothmuller Page 7, Lines 27-31 & Page 8, Lines 1-3). Figure 1, Image area 100 
provides a media view (Rothmuller Figure 1). 

"wherein the time bar has one or more time levels, wherein the display of at 
least one time level is divided into a plurality of segments of time, and wherein 
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the display of each segment of time of the plurality of segments of time of at least 
one time level is based upon the amount of media files associated with the 
segment of time" as the timeline 250 can be used by itself to find all photos taken 
between Jan. 1 , 2000 and Feb. 28, 2000 by moving the adjustable time bands 251 to 
these two respective dates (Rothmuller Page 8, Lines 8-10 & figure 1 and 3). Figure 3 
has levels of year's, months, and days. In figure 3 there are segments of time and their 
size depends upon the amount of files for that time segment. 

"activating one or more time levels of the time bar to display the specific 
predefined time for which a media file is associated" as the timeline 250 can be 
used by itself to find all photos taken between Jan 1 , 2000 and Feb 28, 2000 by moving 
the adjustable time bands 251 to these two respective dates (Rothmuller Page 8, Lines 
8-10). The time levels of timeline shown in the reference has 2000 as year level, Jan as 
month level and 1®' as day level and these levels are being activated to retrieve media 
files. 

"activating the specific period of time to display a representation of the 
media file and the associated predefined time" as the timeline 250 can be used by 
itself to find all photos taken between Jan 1 , 2000 and Feb 28, 2000 by moving the 
adjustable time bands 251 to these two respective dates (Rothmuller Page 8, Lines 8- 
10). 

"selecting the representation of the media file to access the media file" as 

the timeline 250 can be used by itself, or with other tags 350 to specify the criteria used 
to search for matching photos (Rothmuller Page 7, Line 31 & Page 8, Lines 1-2). 
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Rothmuller teaches the elements of claim 16 as noted above but does not 
explicitly teaches "bar divided into segments of unit time and having length along 
the time bar." 

However, Parlter teaches "bar divided into segments of unit time and having 
length along the time bar" as he number of histogram bins that can be clearly 
displayed and the desired start and end date of the histogram timeline, new date bin 
sizes are calculated. In step 238, a new histogram timeline is generated with the 
modified bin sizes. Finally, in step 240, the modified histogram timeline with more 
visible bin size is displayed (Parl^er Paragraph 0048). Parker's timeline is being divided 
into units of time as being days, months, or years. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teaching of the cited references because Parlter's 
teaching would have allowed Rothmuller to provide an effective method for organizing 
representations of digital multimedia objects to facilitate the selection of desired digital 
multimedia objects by selecting a portion of the histogram timeline for viewing such 
thumbnail representations of visual digital objects corresponding to such selected 
portion. 

These references disclose the claimed invention except for segments of time 
having length along time bar (horizontally) which depends on the amount of files 
associated with that length. Prior art teaches vertical bars, where bars represent the 
amount of media files associated with that time unit. It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify the vertical 
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graphs for a specific time segment to horizontal graph along the time bar, since it has 
been held that the provision of adjustability, where needed, involves only routine skill in 
the art. In re Stevens. 101 USPQ 284 (CCPA 1954). 

Further, It would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify the vertical graphs for a specific time segment to 
horizontal graph along the time bar, since it has been held that a mere reversal of the 
essential working parts of a device involves only routine skill in the art. In re Einstein, 8 
USPQ 167. 

With respect to claim 17, Rothmuller teaches "the method of claim 16, 
wherein the step of activating one or more time levels of the time bar to display 
the specific predefined time for which a media file is associated further 
comprises activating one or more time levels of the time bar chosen from the 
group consisting of month level, week level and day level to display the specific 
predefined time for which a media file is associated" as the timeline 250 can be 
used by itself to find all photos taken between Jan 1 , 2000 and Feb 28, 2000 by moving 
the adjustable time bands 251 to these two respective dates (Rothmuller Page 8, Lines 
8-10). The time levels of timeline shown in the reference has 2000 as year level, Jan as 
month level and 1^' as day level and these levels are being activated to retrieve media 
files. 
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With respect to claim 18, Rothmuller teaches "the method of claim 16, 
wherein the step of activating the specific predefined time to display a 
representation of the media file and the associated predefined time further 
comprises activating a specific date to display a representation of the media file 

and the date" as the timeline 250 can be used by itself to find all photos taken between 
Jan 1 , 2000 and Feb 28, 2000 by moving the adjustable time bands 251 to these two 
respective dates (Rothmuller Page 8, Lines 8-10). 

4. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Rothmuller et al. (International Pub No. WO 02/057959 A2). 

With respect to claim 12, Rothmuller teaches "the digital device of claim 11, 
the computer-readable program instructions further comprising a third 
instructions configured to generate a calendar view that represents time in 

calendar format, associates events with respective periods of time" as in addition 
to timelines, the temporal distribution of objects in the database can be represented in a 
calendar view such that the days of the calendar indicate the number of objects having 
metadata associated with a given day of the week in a given week of the month 
(Rothmuller Page 3, Lines 10-16). In addition tags in the events category can include 
pre-defined calendar event such as New Years Eve, and customized calendar events 
such as birthdays and anniversaries (Rothmuller Page 5, Lines 21-23). 
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"a display in communication with the processing unit that presents a 
combined view of the media view and the time bar" as figurel , which shows the 
combined view of both the media view and the time bar (Rothmuller Figure 1 ). 

Rothmuller teaches the elements of claim 12 as noted above but does not 
explicitly teach "and is presented by the display in combination with the media 
view and the time bar." 

However, Rothmuller teaches "and is presented by the display in 
combination with the media view and the time bar" as figurel , which shows the 
combined view of both the media view and the time bar (Rothmuller Figure 1 ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited reference because these 
teachings would have provided the display of all the files and events associated with a 
specific data by the displaying calendar events with combination to media view and time 
bar, thereby limiting the search time of different files and events. 

Response to Arguments 

5. Applicant's arguments filed on 12/19/2007 have been considered but are moot in 
view of the new ground(s) of rejection. 

Examiner has withdrawn the 103 rejections of claims over Rothmuller in view of 
Chao. The new grounds presented in this office action include 103 rejections of claims 
over Rothmuller in view of Parker.. 

See above rejections for the arguments. 
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Claims must be given the broadest reasonable interpretation during examination and 
limitations appearing in the specification but not recited in the claim are not read into the claim 
(See M.P.E.P. 2111 [R-l]). 

Conclusion 

6. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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Contact Information 



7. Any inquiry concerning this communication or earlier communications from tlie 
examiner should be directed to Usmaan Saeed whose telephone number is (571)272- 
4046. The examiner can normally be reached on M-F 8-5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Hosain Alam can be reached on (571)272-3978. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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